Multi-View Picking: Next-best-view Reaching for Improved Grasping in Clutter

Camera viewpoint selection is an important aspect of visual grasp detection, especially in clutter where many occlusions are present. Where other approaches use a static camera position or fixed data collection routines, our Multi-View Picking (MVP) controller uses an active perception approach to choose informative viewpoints based directly on a distribution of grasp pose estimates in real time, reducing uncertainty in the grasp poses caused by clutter and occlusions. In trials of grasping 20 objects from clutter, our MVP controller achieves 80% grasp success, outperforming a single-viewpoint grasp detector by 12%. We also show that our approach is both more accurate and more efficient than approaches which consider multiple fixed viewpoints.Comment: ICRA 2019 Video: https://youtu.be/Vn3vSPKlaEk Code: https://github.com/dougsm/mvp_gras

Towards adaptive and autonomous humanoid robots: from vision to actions

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions

Tuning Modular Networks with Weighted Losses for Hand-Eye Coordination

This paper introduces an end-to-end fine-tuning method to improve hand-eye coordination in modular deep visuo-motor policies (modular networks) where each module is trained independently. Benefiting from weighted losses, the fine-tuning method significantly improves the performance of the policies for a robotic planar reaching task.Comment: 2 pages, to appear in the Deep Learning for Robotic Vision (DLRV) Workshop in CVPR 201

Visual Servoing from Deep Neural Networks

We present a deep neural network-based method to perform high-precision, robust and real-time 6 DOF visual servoing. The paper describes how to create a dataset simulating various perturbations (occlusions and lighting conditions) from a single real-world image of the scene. A convolutional neural network is fine-tuned using this dataset to estimate the relative pose between two images of the same scene. The output of the network is then employed in a visual servoing control scheme. The method converges robustly even in difficult real-world settings with strong lighting variations and occlusions.A positioning error of less than one millimeter is obtained in experiments with a 6 DOF robot.Comment: fixed authors lis

Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies

Various approaches have been proposed to learn visuo-motor policies for real-world robotic applications. One solution is first learning in simulation then transferring to the real world. In the transfer, most existing approaches need real-world images with labels. However, the labelling process is often expensive or even impractical in many robotic applications. In this paper, we propose an adversarial discriminative sim-to-real transfer approach to reduce the cost of labelling real data. The effectiveness of the approach is demonstrated with modular networks in a table-top object reaching task where a 7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter through visual observations. The adversarial transfer approach reduced the labelled real data requirement by 50%. Policies can be transferred to real environments with only 93 labelled and 186 unlabelled real images. The transferred visuo-motor policies are robust to novel (not seen in training) objects in clutter and even a moving target, achieving a 97.8% success rate and 1.8 cm control accuracy.Comment: Under review for the International Journal of Robotics Researc

Role of vibrational entropy in the stabilization of the high-temperature phases of iron

The phonon dispersions of the bcc and fcc phases of pure iron ({\alpha}-Fe, {\gamma}-Fe and {\delta}-Fe) at ambient pressure were investigated close to the respective phase transition temperatures. In the open bcc structure the transverse phonons along T1 [{\xi}{\xi}0] and T1 [{\xi}{\xi}2{\xi}] are of particularly low energy. The eigenvectors of these phonons correspond to displacements needed for the transformation to the fcc {\gamma}-phase. Especially these phonons, but also all other phonons soften considerably with increasing temperature. Comparing thermodynamic properties of the fcc and the two bcc phases it is shown that the high temperature bcc phase is stabilized predominantly by vibrational entropy, whereas for the stabilization of the fcc phase electronic entropy provides an equal contribution.Comment: to appear in Physical Review

EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation

We present the Evolved Grasping Analysis Dataset (EGAD), comprising over 2000 generated objects aimed at training and evaluating robotic visual grasp detection algorithms. The objects in EGAD are geometrically diverse, filling a space ranging from simple to complex shapes and from easy to difficult to grasp, compared to other datasets for robotic grasping, which may be limited in size or contain only a small number of object classes. Additionally, we specify a set of 49 diverse 3D-printable evaluation objects to encourage reproducible testing of robotic grasping systems across a range of complexity and difficulty. The dataset, code and videos can be found at https://dougsm.github.io/egad/Comment: IEEE Robotics and Automation Letters (RA-L). Preprint Version. Accepted April, 2020. The dataset, code and videos can be found at https://dougsm.github.io/egad

Geochemische und petrologische Untersuchungen der Ola- und Hypotetica- Plateaubasalte im Okhotsk- Chukotka Volcanic Belt in Russland (Nord- Ost Sibirien)

Der vulkanische Gürtel von Okhotsk- Chukotka (OCVB) dehnt sich über die kontinentalen Ränder vom Nordosten Russlands mit einer Länge von über 3000 km aus und repräsentiert den längsten Anden ähnlichen magmatischen Bogen im Pazifik. Die Mehrheit des OCVB besteht aus subduktionsbezogenem kalk-alkalischen Plutoniten und Vulkaniten, die Granitoide, andesitische Basalte, Andesite, Dacite, Rhyolithe, Tuffe, klastische Gesteine aus Konglomerat und Sandstein beinhalten. Die letzten vulkanischen Ereignisse werden durch leicht abtauchende campanische hoch Kalium- bis kalk-alkalische Basalte repräsentiert. Über die Verweilzeit der magmatischen Aktivität des OCVB’s wird noch debattiert, aber im allgemeinen wurde sie von Albian bis Campanian eingeteilt. Wir untersuchten letztere vulkanische Abschnitte anhand von basaltischen Längsprofilen, die als Ola- Plateau Basalte (OPB) und Hypotetica- Plateau Basalte (HPB) bezeichnet werden. Beide Plateaus bestehen hauptsächlich aus massiven basaltisch- andesitischen, trachy- basaltischen, basaltisch- trachyandesitischen Laven und einigen sich quererstreckenden rhyolitischen Gängen. Es wird vermutet, dass diese Gesteine die letzte vulkanische Aktivität zeigen, die durch die Subduktion der paläo- pazifischen Platte in dieser Region verursacht wurde. Die ausgestossenen Laven der OPB besitzen eine Mächtigkeit von ungefähr 0.5 km und einem abgeschätzten Volumen von 222 km³. Feinkörnige bis zu 4 m dicke rhyolitische Gänge zeigen das allerletzte Ereignis in diesem untersuchten Gebiet. Bezogen auf 40Ar/39Ar (Gesteinsbildende Minerale: Plagioklas, Sanidin, Biotit, Hornblende und Gesamtgestein) und U/Pb (Zirkon) Datierungen (Hourigan, Akinin, 2004; Tikhomirov et al., 2006), ist das durchschnittliche Alter der OPB und HPB 78.8- bis 74 Millionen Jahre alt. Die basaltischen Gesteine, die das Ola- Plateau aufbauen, sind vor allem feinkörnige hoch Kalium- bis kalk-alkalische Basalte mit zwei Pyroxentypen-, Plagioklas- und stark bis gemäßigt umgewandelten Olivin- Phänokristallen. Die Magnesiumnummer der hoch Kalium- bis kalk-alkalische Basalte variiert von 0.35- bis 0.57 Gew. % und TiO2 von 1.2- bis 2,3 Gew. % während CaO mit zunehmendem MgO positiv korreliert. Die OPB- und HPB Laven, normiert auf den primitiven Erdmantel entsprechenden Spurenelemente, können in zwei Gruppen unterteilt werden: Gruppe (1) wird durch positives Strontium im Bezug zu seinen benachbarten Elementen und Gruppe (2) durch negatives Strontium, charakterisiert. Auffallend bei beiden Gruppen sind die starken negativen Nb und Ta und die positiven Ba und Pb Anomalien. Der Trog von Strontium in der Gruppe (2) wird begleitet durch eine negative Eu Anomalie aufgrund unterschiedlicher Grade der Plagioklasfraktionierung. Durch erhöhte LILE Konzentrationen und markante negative Nb und Ta Anomalien, kann man davon ausgehen, das die OPB und HPB Laven von aufsteigenden Fluiden der subduzierenden paläo- pazifischen Platte beeinträchtigt wurden. Die niedrige Mg# der untersuchten Proben und eindeutige fraktionierte Kristallisation in geringer Tiefe in der Kruste (Lavas mit negativer Eu- Anomalie) unterstützen die Annahme, dass die untersuchten Laven im Zusammenhang mit subduktions bezogenen Fluiden, durch Krustenkontamination beeinflusst wurden.The Okhotsk- Chukotka volcanic belt (OCVB) stretches along the continental margin of northeast Russia on about 3000 km, and represents the largest Andean-style magmatic arc throughout northern Pacifica. The majority of OCVB composed by subduction-related Cretaceous calc-alkaline plutonic and volcanic rocks including granitoids, andesitic basalts, andesites, dacites, rhyolites, and their tuffs, rare beds of non- marine clastic rocks with conglomerates and sandstones. Final stages of volcanism are represented by gently dipping Campanian high-K to calc-alkaline basalts. The duration of the magmatic activity in OCVB is still in debate but generally it has been estimated from middle of Albian to Campanian. We studied final volcanic episodes of belt croped out as basalt cross-sections and called as the Ola- Plateau Basalts (OPB) and the Hypotetical Plateau Basalts (HPB). Both plateaus composed mostly by massive basaltic andesites, trachy- basaltic, basaltic trachy- andesitic lavas and rare cross-cutted rhyolitic dykes. The rocks represent the final volcanic activity believe to be related to the subduction of the palaeo-Pacific plate in this region. The exposed lavas of OPB have a thickness of 0.5 km and the estimated volume is about 222 km³. Fine grained 4 m thick rhyolitic dykes represent the very last event of the studied sequence. According to 40Ar/39Ar dating of the rock forming minerals: plagioclase, sanidine, biotite, hornblende and whole rock and U/Pb-zircon dating (Hourigan, Akinin, 2004; Tikhomirov et al., 2006), the average age of the OP-/ HPB is 78.8 to 74 Ma. The basaltic rocks that build up the Ola- Plateau are mainly fine grained high-K calc- alkaline basalts with two types of pyroxene, plagioclase and strongly to moderately altered olivine phenocrysts. The Mg# of the high-K to calc- alkaline basalts vary from 0.35- to 0.57 and the TiO2 from 1.2- to 2.3 Gew. %, whereas CaO correlates positive with MgO, contents. The OPB and HPB lavas, according to their primitive mantle normalized trace elements, can be divided into two groups: Group (I) is characterized by positive Sr anomaly with respect to the neighbour elements, group (II) has negative Sr anomaly. Common features of all two groups are the prominent negative Nb and Ta anomalies and the positive Ba and Pb anomalies. The trough at Sr in the group (II) patterns is associated with negative Eu anomalies attributed to the variable degrees of plagioclase fractionation. As can be inferred from the elevated LILE concentrations and the prominent negative Nb and Ta anomalies, the OPB and HPB lavas clearly have been affected by fluids released from the subducted paleo-Pacific plate. However, the low mg# of the studied lavas and the clear evidence for fractionational crystallization at shallow depths in the crust (lavas with negative Eu-anomalies) support the possibility that in addition to the introduction of subduction related fluids, crustal contamination could have affected the studied lavas

Evaluating task-agnostic exploration for fixed-batch learning of arbitrary future tasks

Deep reinforcement learning has been shown to solve challenging tasks where large amounts of training experience is available, usually obtained online while learning the task. Robotics is a significant potential application domain for many of these algorithms, but generating robot experience in the real world is expensive, especially when each task requires a lengthy online training procedure. Off-policy algorithms can in principle learn arbitrary tasks from a diverse enough fixed dataset. In this work, we evaluate popular exploration methods by generating robotics datasets for the purpose of learning to solve tasks completely offline without any further interaction in the real world. We present results on three popular continuous control tasks in simulation, as well as continuous control of a high-dimensional real robot arm. Code documenting all algorithms, experiments, and hyper-parameters is available at https://github.com/qutrobotlearning/batchlearning